Maximization of Empirical Shannon Information in Testing Significant Variables of Linear Model

Search for an unknown set A; Card(A) = s, of signiicant variables of a linear model with random IID discrete binary carriers and nitely supported IID noise is studied. Two statistics T 1 ; T s ; based on maximization of Shannon Information (SI) of the corresponding classes of joint empirical input-output distributions , are proposed inspired by the related study in Csiszar and KK orner (1981). The rst one compares sequences of values of each variable and of the output separately. The second one explores the relation between the subsets of the (N t) design matrix corresponding to each subset of variables of given car-dinality and the output sequence. Here N is the number of experiments and t is the total number of variables. Both statistics are shown to be asymptotically as eecient as the ML-test for the corresponding classes of joint empirical distributions in the artiicial case when ML-test is applicable: if the unknown parameters b ; 2 A; of the model and the distribution of errors are known. Our tests do not require this information. Therefore, they are asymptotically uniformly most eecient in the corresponding classes of tests. The second statistic is shown to provide asymptotically best rate of search for the set A of signiicant variables when t ! 1; but requires about t s log t cycles of computing. This may appear in accessible for actual computations in some applications. The rst statistic requires only t log t cycles of computing operations and provides the best order of magnitude of the characteristics studied for the second class of tests.